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INTRODUCTION 

The  purpose  of  this  study  is  to  develop,  implement  and  evaluate  a  web-based,  self-paced 
training  program  to  provide  skill-improvement  oriented  continuing  education  for  mental  health 
professionals.  It  is  intended  to  reach  a  broad  range  of  VA  health  professionals  presenting 
content  that  is  highly  relevant  and  easy  to  access  for  practitioners.  It  follows  adult  learning 
principles  and  incorporates  interactivity,  practice,  and  modeling  of  effective  treatment  for 
Posttraumatic  Stress  Disorder  (PTSD).  Program  evaluation  will  include  a  multi-dimensional 
assessment  of  cognitive,  attitudinal  and  skills-based  aspects  of  the  CBT  and  motivational 
interviewing  components.  A  randomized  design  will  be  used  in  which  120  participants  will  be 
randomly  assigned  to  one  of  three  conditions:  (i)  one  group  (n=40)  will  receive  web-based 
training  plus  consultation,  (ii)  a  second  group  (n=40)  will  receive  web-based  training  without 
consultation,  and  (iii)  a  third  control  group  (n=40)  will  receive  neither  the  web-based  training  nor 
consultation.  The  effectiveness  of  training  will  be  evaluated  by  means  of  simulated  patient 
interviews  and  on-line  assessment  of  core  skills  (primary  outcome),  knowledge,  attitudes  and 
self-efficacy  (secondary  outcomes).  The  development  of  this  training  program  involves  the 
expertise  of  two  research  partners  with  extensive  knowledge  and  prior  experience  in  PTSD, 
CBT,  and  development  of  educational  training  programs.  The  VA’s  National  Center  for  PTSD 
(NCPTSD)  has  designed  dozens  of  training  programs  in  the  past  for  mental  health  clinicians  on 
PTSD,  and  the  New  England  Research  Institutes,  Inc.  (NERI),  a  small  business  research  firm 
that  has  designed  a  comprehensive,  skills-oriented  CBT  training  program  as  well  as  numerous 
other  award  winning  CE  and  CME  programs  for  health  care  practitioners. 

BODY 

Task  T.  Create  web-ready  text  for  core  components  of  PTSD-related  CBT,  drawing  on  all 
relevant  training  materials  currently  being  used  in  the  NCPTSD. 

During  the  first  year  of  the  project,  the  content  development  task  group  has  completed 
development  of  the  three  modules  for  the  web-based  training  program  (motivational 
enhancement,  goal-setting  and  behavioral  task  assignment).  Furthermore,  the  content  of  the 
training  modules  have  been  programmed  and  are  currently  accessible  on  a  live  website 
(website  is  currently  under  beta  testing  status).  A  team  of  carefully-selected  expert  content 
developers  was  assembled,  comprised  of  the  project  investigators,  in  addition  to  Amy  Naugle, 
Ph.D.,  Jennifer  Sharpe  Potter,  Ph.D.,  J.  Gayle  Beck,  Ph.D.,  and  Brett  T.  Litz,  Ph.D.  These  latter 
individuals  were  invited  to  serve  as  special  consultants  to  the  project  based  on  their  extensive 
experience  and  knowledge  in  the  specific  content  areas. 
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Task  2:  Translate  this  text  into  an  innovative  learning  program  that  blends  learner-paced, 
learner-initiated  mastery  of  new  materials  using  web-delivered  interactive  content. 

Web  programming  for  the  training  modules  has  been  completed  and  is  currently  in  the 
beta  testing  stage.  The  official  launch  of  the  website  is  planned  for  September,  2009. 

Task  3:  Develop  evaluation  materials  to  measure  knowledge  and  self-efficacy  of  CBT  skills 
pre/post  intervention. 

The  evaluation  design  for  the  study  has  been  approved  by  the  local  IRBs  (Stanford  and 
NERI)  and  is  now  being  prepared  for  implementation.  The  study  design  includes  a  3-arm  design 
with  a  target  sample  size  of  120  (n=40  Web  training  with  consultation;  n=40  Web  training  only; 
n=40  Control).  See  Figure  1. 

•  Data  Collection:  There  are  several  points  of  data  collection  as  defined  in  Figure. 

■  Accessing  the  Sample:  Postcards  and  email  invitations  have  been  developed. 
Postcards  have  been  distributed  at  one  national  meeting  and  a  second  set  will  be 
distributed  at  conferences  and  other  relevant  in-person  meetings  closer  to  the 
time  of  recruitment.  Email  invitations  will  begin  to  be  sent  out  in  batches  when 
recruitment  begins  (September,  2009).  A  registration  site  is  available  for 
interested  individuals  to  access  and  provide  an  email  address  for  recruitment. 

■  Web  based  Screening  Instrument:  Interested  users  will  access  an  online 
screening  instrument  to  determine  eligibility.  Those  eligible  will  be  directed  to  an 
online  informed  consent  and  registration  process.  Those  not  eligible  will  be 
thanked  for  their  interest  and  informed  that  they  are  not  eligible  for  the  study. 

■  Online  informed  consent:  The  terms  of  the  study  will  be  clearly  defined  with  an 
‘accept’  (or  decline)  participation  button.  Those  who  accept  will  move  to  the  next 
phase  of  the  study.  Those  who  decline  will  be  thanked  and  informed  that  they 
have  opted  not  to  continue  (the  consent  form  has  been  approved  by  both  the 
Stanford  and  NERI  IRBs). 

■  Online  registration:  This  form  includes  basic  demographics,  type  of  practitioner 

(for  CE/CME  accreditation),  number  of  years  practiced,  geographic  location,  etc. 

■  Pretest  Evaluation:  Once  registration  is  completed,  users  will  be  directed  to  an 

approximately  30-40  minute  web-based  pretest,  which  has  been  finalized  and  is 
currently  being  reviewed  by  the  Stanford  and  NERI  IRBs. 
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■  Simulated  Patient  Interview  1:  Following  completion  of  the  pretest  evaluation, 
the  data  management  system  will  identify  participants  as  requiring  an 
appointment  with  a  simulated  patient.  The  interview  will  be  tape  recorded  and 
the  simulated  patient  will  record  date  and  time  of  completion.  Completion  date 
will  be  entered  into  the  data  management  system  to  trigger  randomization.  The 
assessment  form  will  be  identified  as  ‘pending’  and  will  also  be  data  entered. 

■  Randomization  :  Once  the  participant  has  been  entered  into  the  system  as 

having  completed  the  interview,  they  will  be  randomized  to  one  of  the  three  arms. 

If  selected  for  either  of  the  web  arms,  a  user  name  and  password  will  be 
provided  for  the  web  course^ 

■  Completing  the  Course:  The  online  participants  will  have  a  4  week  window  to 
complete  the  online  training  course  at  their  pace  in  any  order  before  being 
prompted  to  take  the  mid-point  evaluation.  After  the  evaluation  has  been 
completed,  the  course  will  be  available  to  online  participants  again  for  use  in 
daily  practice. 

■  Mid-point  Evaluation:  Following  the  course  review  or  completion  of  the  four  week 

period,  participants  will  access  a  CE/CME  quiz  and  mid-point  evaluation.  The 
control  group  will  be  sent  an  automatic  email  to  complete  a  midpoint  quiz 
(knowledge  and  self-efficacy  only,  no  feedback  on  the  web  site). 

■  Consultation  :  For  the  Web  plus  Consultation  group,  this  will  also  trigger  the 

scheduling  of  consultation  calls.  The  window  for  completing  the  consultation 
activities  is  6  weeks.  Consultants  will  be  recruited  from  the  National  Center  of 
PTSD’s  staff  of  experienced  clinicians. 

■  Post  Test  Evaluation:  Once  the  6  week  window  is  closed  (a  total  of  10  weeks  for 
the  control  participants),  the  study  participants  in  all  three  groups  will  be  sent  an 
email  directing  them  to  the  post  test,  which  is  also  web-based.  This  time  period 
will  be  standardized  across  conditions. 

■  Simulated  Patient  Interview  2:  After  the  post-test  is  completed,  the  data 
management  system  will  identify  participants  as  requiring  their  second 
appointment  with  a  simulated  patient. 

■  Rater  Forms:  Following  completion  of  the  interview,  the  audiotape  will  be 

transcribed  and  reviewed  for  skills  rating  by  the  interview  rating  staff.  The 
completion  date  for  the  interview  will  be  entered  into  the  data  management 
system  and  the  rater  form  rater  forms  will  also  be  data  entered.  At  this  time, 
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CE/CME  credits  will  be  awarded  to  participants.  Control  participants  will  also  be 
given  access  to  the  web  site  once  participation  is  completed. 

Figure  1. 

Diagrammatic  Representation  of  Evaluation  Design. 


6  weeks 


Task  4\  Develop  simulated  case  for  evaluation  of  clinical  skills  and  train  actors  to  participate  in 
the  simulated  patient  evaluation. 

During  the  first  year  of  the  study  5  actors  have  been  selected  to  play  the  role  of  simulated 
patients  for  the  evaluation  interviews  prior  to  and  post  training.  A  training  manual  was 
developed  for  the  actors  which  provided  them  the  procedures  of  the  interviews,  scripts,  decision 
rules,  and  an  overview  of  the  simulated  patient’s  character.  Actors  have  been  trained  during  a 
two-day  workshop  that  focused  on  the  personality  of  the  simulated  patient,  response  patterns, 
decision  rules,  and  levels  of  resistance.  To  limit  the  variability  of  evaluation  scores,  consistency 
across  the  actors  was  the  primary  focus  of  the  training,  as  multiple  actors  are  needed  for 
scheduling  purposes,  and  having  more  than  one  actor  increases  the  probability  of  variation 
across  actors. 
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Task  5:  Filing  of  CE/CME  applications 

Currently,  obtaining  Continued  Education  (CE/CME)  accreditation  for  prospective  study 
participants  is  near  completion  and  should  be  obtained  in  August,  2009.  CME  accreditation  is 
being  sought  from  two  potential  providers. 

Task  6:  Filing  of  local  IRB  applications  (Stanford  and  NERI  IRBs) 

Submission  to  local  IRBs  (Stanford  and  NERI)  with  finalization  of  the  study  protocol  was 
completed  early  in  Quarter  3.  Submission  to  local  and  DOD  IRBs  of  all  study  materials  is 
currently  in  process.  Study  recruitment  is  on  pace  to  begin  in  September,  2009. 

Task  7:  Evaluate  telephone  consultation  designed  to  ensure  that  the  CBT  skills  learned  are 
implemented  by  training  participants  in  routine  clinical  care  in  a  competent  manner. 

Currently,  recruitment  is  on  pace  to  begin  in  September,  2009,  pending  final  Stanford, 
NERI,  and  DoD  IRB  approvals  of  all  study  materials.  We  anticipate  receiving  these  final 
approvals  by  early  September,  2009.  Specific  details  of  all  study  activities  are  described  in  this 
report. 

Task  8:  Measure  the  feasibility  of  the  methodology  and  content  of  the  training  program. 

Currently,  recruitment  is  on  pace  to  begin  in  September,  2009,  pending  final  Stanford, 
NERI,  and  DoD  IRB  approvals  of  all  study  materials.  We  anticipate  receiving  these  final 
approvals  by  early  September,  2009.  Specific  details  of  all  study  activities  are  described  in  this 
report. 

Task  9:  Conduct  a  rigorous  research  study  on  the  effectiveness  of  these  training  dissemination 
methods  relative  to  the  local  training  practices  currently  in  place. 

Currently,  recruitment  is  on  pace  to  begin  in  September,  2009,  pending  final  Stanford, 
NERI,  and  DoD  IRB  approvals  of  all  study  materials.  We  anticipate  receiving  these  final 
approvals  by  early  September,  2009.  Specific  details  of  all  study  activities  are  described  in  this 
report. 

Key  Research  Accomplishments 

•  Recruitment  and  training  of  the  project  team  (including  project  staff  and  consultants) 

•  Development  of  the  study  protocol 

•  Development  of  the  study  evaluation  materials 


•  Development  and  pre-testing  of  core  CBT  content  modules 

•  Development  and  pre-testing  of  the  web  programs  for  content  delivery 

•  Development  of  a  simulated  patient  case  for  evaluation  of  clinical  skills 

•  Recruitment  and  training  of  actors  to  participate  in  the  simulated  patient  evaluation 

•  Filing  of  local  IRB  applications  (Stanford  and  NERI  IRBs) 

•  Filing  of  CE/CME  applications 

Reportable  Outcomes 

At  this  time  there  are  no  reportable  outcomes  as  recruitment  for  the  study  has  not  yet 
begun.  Recruitment  has  been  planned  to  begin  in  September,  2009. 

Conclusion 

The  first  year  of  the  study  has  yielded  significant  results:  Protocol  finalization  and  approval, 
training  modules  developed,  finalized  and  integrated  to  the  web-based  training  program,  and 
development  and  finalization  of  the  evaluation  materials.  The  work  planned  for  year  2  of  the 
study  is  summarized  as  follows: 

1)  Obtain  IRB  approval:  Final  IRB  approval  of  all  study  materials  is  needed  at  the  local 

(Stanford  and  NERI)  and  DoD  levels  prior  to  initiation  of  the  study.  Currently,  training  and 
evaluation  materials  have  been  submitted  to  both  of  the  local  IRBs.  After  approval  by  the 
local  IRB’s,  submission  to  the  ORP  will  occur  for  final  approval. 

2)  Obtain  CE/CME  Accreditation:  Since  study  participants  cannot  be  compensated  for  their 
time  with  anything  of  monetary  value,  CE/CMEs  will  be  given  out  for  completion  of  the  web- 
based  training.  CE/CME  accreditation  for  the  study  is  well  underway.  The  site  has  been 
reviewed  by  both  the  CME  and  Social  Work  course  directors  and  forms  are  pending  final 
approval.  These  activities  will  be  completed  early  in  year  two,  prior  to  study  recruitment. 

3)  Launch  of  web-based  training  program:  The  web-based  training  program  is  currently 
accessible  on  a  live  website;  however,  beta  testing  is  still  underway.  The  official  launch  of 
the  website  is  currently  planned  for  September,  2009. 

4)  Begin  study  recruitment:  Study  recruitment  will  begin  with  the  launch  of  the  web-based 
training  program  in  September,  2009. 
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